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Abstract 

This study aimed to find out whether watching subtitled cartoons influences incidental 
vocabulary learning. The study was conducted with 42 first grade English Language Teaching 
(ELT) department students at the University of Mehmet Akif Ersoy, Burdur. To collect data 
from the subjects, a 5-point vocabulary knowledge scale was used and 18 target words were 
integrated into the scale. The pre-test and post-test group design was selected for the 
administration. After subjects had been randomly assigned into two groups (one subtitle group 
and the other no-subtitle group), they were given the same pre- and post-tests. 

The findings of study did not support the assumption that the subtitle group would 
outperform the no-subtitle group, since there were no significant differences between two 
groups according to t-test results. However, there was significant improvement in both of the 
groups from pre-test to post-test scores. This progress was attributed to the presentation of 
target words in cartoons. In this way, the target words were contextualized and it became easy 
for participants to elicit the meanings of the words. 

Keywords: animated cartoons, vocabulary development, subtitles and captions, 
language learning 


1. Introduction 

As a consequence of the rapid development in science, technology and media, foreign/second 
language teaching field has gained much improvement in the recent years. Especially, the 
function of language classes has changed dramatically. Their potential to provide 
comprehensible input, which, according to Krashen (1982), leads to ‘subconscious language 
acquisition’, has increased a lot with the provision of such technical devices as TV, LCD 
projector, laptop, DVD player and video materials into the classrooms. Out of all these, 
particularly, the use of video and TV has grown rapidly due to the increasing focus on 
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communicative aspects of language use (Cakir, 2006). This has helped language teaching 
quality boost both in and outside the class. However, language learners have difficulty in 
following and understanding video materials and TV programs in the target language. 
Therefore, such visual materials have been enhanced with subtitles either in the mother 
tongue or the target language in order to facilitate the comprehensibility of these materials. 

The study attempts to explore the impact of animated cartoons enhanced with English 
subtitles with upper intermediate first-year ELT students. Particularly, the research aims to 
answer the question “How would a language material (i.e., animated cartoons) with or without 
English language subtitles affect these students’ vocabulary development?” It is assumed that 
significant differences will be found between the groups watching animated cartoons with and 
without subtitles in developing vocabulary knowledge. 

This study differs from the previous studies in that it focuses on vocabulary 
development process, rather than on reading or listening comprehension. While doing this, it 
avoids using discrete multiple-choice tests, used extensively in the previous studies. Instead, a 
vocabulary knowledge scale (VKS) developed by Wesche & Paribakht (1996) was applied as 
this scale is process-oriented and compatible with the view that vocabulary acquisition is a 
continuum of development. 

Another innovation in the study is using animated cartoons as language input. To the 
researcher’s best knowledge, no other study has used animated cartoons as a material in 
similar studies yet. To understand the matter well, it is better to have a look at the previous 
literature and related studies about subtitles and captions in audiovisual materials 

2. Literature review 

2.1. Subtitles as language materials 

There have been a great number of supporters of the use of subtitles in videos and TV 
programs for various reasons. For example, Danan (2004) claims that audiovisual materials 
enhanced with captions or subtitles may function as a powerful educational tool in many 
ways. For example, 

(1) they improve the listening comprehension skills of second/foreign language learners; 

(2) facilitate language learning by helping students visualize what they hear and 

(3) increase language comprehension and lead to additional cognitive benefits, such as greater 

depth of processing (p. 67). 

However, Danan (2004) also reports that many language teachers are against their use in 
audiovisual materials. This might be because they fear that subtitles may distract learners’ 
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attention, especially that of lower-level learners, from the actual spoken language to written 
text and create a sense of laziness on the part of students (Taylor, 2005). 

There are also several studies that focused on the effects of subtitles/captions on 
comprehension in reading, listening and vocabulary acquisition. These studies aimed to 
investigate whether captioned videos or TV programs are more effective than non-captioned 
ones (Baltova, 1999; Danan, 1992, 2004; Garza, 1991; Markham, 1993, 1999; Neuman & 
Koskinen, 1992). The general findings of these studies supported the common assumption 
that subtitles and captions are powerful instructional tools in learning vocabulary and 
improving reading and listening comprehension skills of language learners. However, 
according to Winke, Gass, and Sydrenko (2010), it is difficult to generalize the findings of the 
studies reviewed above for at least two reasons: “First, several studies did not group subjects 
by proficiency levels; second, the types of tests used to measure the effects of language 
learners’ processing of captions varied widely” (p.67). Hence, the differences in 
comprehension may have resulted from the effects of captions/subtitles or from the level of 
proficiency, and we still do not know whether the other type of tests may produce similar 
results or not. 

In her experiment with keyword captions, full text captions and no-text groups, 
Guillory (1998) demonstrates that “the keyword captions group outperformed the no text 
group and that the full text captions group outperformed the keyword captions group” (p. 89). 
Rather than focusing attention on reading and comprehension skills, she attempted to use 
captions to help learners link written words with their phonetic realizations, namely; they 
might arouse the phonological visualization of aural cues in the minds of listeners (Bird & 
Williams, 2002). That is another benefit of captions/subtitles in visual materials. 

In addition to this, captions/subtitles play an important role in lowering the affective 
filter, which psychologically affects one’s learning. For example, we feel comfortable since it 
is easy for us to get meaning from foreign language films with subtitles and captions. This 
assumption was supported by Kos ki nen et al., who examined the effects of captioned videos 
on incidental reading vocabulary knowledge and confirmed that captioned videos 
considerably improved the reading vocabulary knowledge of the participants (cited in Yiiksel 
& Tanriverdi, 2009). 

2.2. Incidental vocabulary learning 

Vocabulary, the core of the language, is not acquired at one shot. It necessitates a long 
process. Throughout this process, learners become familiarized with the encountered words. 



Teaching English with Technology, 12(4), 3-15, http://www.tewtiournal.org 


6 


What makes them familiar with words for acquisition is the frequency of their usage and the 
number of encounters in different forms and contexts (Nation, 1990; Schmidt, 2001). Yiiksel 
and Tannverdi (2009) claimed that vocabulary acquisition occurs along a continuum of 
development, as presented in Figure 1. Furthermore, they suggest that a pre-test and a post¬ 
test must be separated by a longer period and conducted in contrast with other learning set¬ 
ups (e.g. other media or means to learn words). 


. > 

Less or little knowledge more or target like knowledge 

Figure 1: A continuum of vocabulary knowledge. 

Another researcher, Fraser (1991) reported that “some incidental L2 vocabulary learning 
occurs in the course of reading for comprehension through the process of inferring word 
meaning” from the context (p. 225). Learning context is really important for vocabulary 
teaching and learning since meanings of some vocabulary items are ambiguous when isolated 
(Sun & Dong, 2004). Even children when mastering their mother tongue learn new 
vocabulary items from different reading and listening situations and contexts (Day, Omura & 
Fliramatsu, 1991). The same is held to be true for foreign language vocabulary learning as 
well. 

2.3. Background to the study 

Having reviewed the past literature, it was noticed that there are not many studies aimed at 
researching the effect of captions and subtitles on vocabulary development in the Turkish 
context. There are few studies that mostly used movie or video clips as materials in their 
experimental treatment. New materials (e.g. animated cartoons, songs) should be integrated 
into studies with a focus on different parts of language. 

Although Turkey is rich in TV channels, there are a limited number of TV channels 
that enhance their programs with subtitles. However, the problem is that channels like E2, 
CNBCE and TNT broadcast their programs either with the Turkish subtitles or in the mother 
tongue. Another source of subtitles for learners are original DVDs, which offer movies with 
many options: “with subtitles in the original language, just the original language, original 
language but with Turkish subtitles”. 
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However, the major source of subtitles is the Internet, and one can download the 
subtitles of any video in the target language. Currently, all of these facilities are available to 
almost any language learners in Turkey. 

3. The study 

3.1. Setting and participants 

The study was conducted with 42 English Language Teaching (ELT) 1 st grade students 
studying at Mehmet Akif Ersoy University (MAKU) in the academic term 2010-2011, 
including 13 male and 29 female students. The participants of the study took intensive 
English courses at high school before their undergraduate education in ELT. Since the 
university does not have a prep-program, they did not take a proficiency test prior to their 
study at the ELT department. However, they all took the LYS 5 exam (the foreign language 
exam for those wishing to study in ELT) to meet the placement requirements of YOK (The 
Council of Higher Education). The exam covered questions in reading comprehension, 
vocabulary and grammar knowledge. All of the participants were approximately at a similar 
English proficiency level (i.e. upper intermediate) based on their scores obtained from the 
LYS 5 exam. 

3.2. Design and procedure 

A pre-test - post-test experiment and group framework was used as a research design in the 
study. The participants were randomly assigned to each group. In group A (the subtitle 
group), participants watched cartoon movies with the English subtitles and in Group B (the 
no-subtitle group) participants watched without subtitles. Both groups were given the same 
pre- and post-tests. Table 1 illustrates the design of study. 

Table 1: Design of the study. 


Pretest 

Treatment 

Post-test 

VKS is given to both groups 

Group A (cartoons with subtitles) 

Group B (cartoons without subtitles) 

VKS is given to both groups 


The administration of the tests and treatments was done in the computer lab, where 
participants often watch movies and videos in English in other classes. An LCD projector and 
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a laptop were available for the treatment in the lab. The free software Gom Player, a media 
player that has the facility to incorporate subtitles into the moving picture, was utilized to play 
the cartoons. 

A popular American television series, Family Guy, was selected as the teaching aid for 
this study. It is a popular animated TV sitcom followed by a very wide audience around the 
world. Episodes 3 and 6 from the 3 season were selected since these episodes do not include 
much slang and are clear for students to understand. Twenty-four target words from these two 
episodes were drawn out based on the proficiency level and background of the students 
following the suggestions of their course lecturer. All of these words belonged to the same 
word category (all verbs). Pilot tests were conducted with similar freshman students from the 
ELT department of Suleyman Demirel University to determine the suitability of the selected 
target words (Cronbach’s Alpha .864). According to the results of the pilot tests, six of the 
verbs that do not serve the purpose of the study were eliminated from the scale. Totally, 
eighteen target words that are not frequently used were kept in the scale. 

Wesche and Paribakht’s (1996) 5-point self-report scale of vocabulary knowledge 
(VKS) was adapted to measure the vocabulary development of the subjects since it allows to 
specify the stages of vocabulary acquisition from first exposure to production and enables the 
researcher to determine how well the participants know these vocabulary items. This scale 
also shows the students’ partial knowledge of items. The VKS is composed of 5 levels as 
follows: 

1:1 don't remember having seen this word before. 

2: I have seen this word before but I don't know what it means. 

3: I have seen this word before and I think it means_(synonym or translation). 

4: I know this word. It means_(synonym or translation). 

5: I can use this word in a sentence, e.g.:_(if you do this section, please 

also do section 4). 


Figure 2: The Vocabulary Knowledge Scale (Wesche and Paribakht, 1996). 

The participants’ responses were simply coded according to the level they chose for 
each word on VKS (Wesche and Paribakht, 1996). However, level 3, 4 and 5 entail some kind 
of production from students. Thus, the researchers checked the accuracy of their answers and 
marked their choices as they are, if they were correct. However, if the responses were 
incorrect, the researcher downgraded the participant’s choice by one level. For instance, if a 
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participant chose Level 3 for a word, the answer was marked as Level 3 only if the response 
was accurate, if it was not, then the researcher marked the answer as Level 2. 

4. Results and findings 

18 target words to collect data were integrated into Wesche’s VKS. Then, the scales were 
distributed to the students in Groups A and B during their course with the researcher at the 
computer lab at different times of the same day. One week after the pretest the students in 
Group A watched episodes 3 and 6 from the 3 season of Family Guy with the English 
subtitles and the ones in Group B watched them without subtitles at different class hours. One 
week later both groups were given the post-tests. 

The expected responses were based on a 5-point vocabulary knowledge scale ranging 
in level from 1 to 5 (see Figure 2 above). To keep the study confidential, the students were not 
informed about the purpose of the study till they finished with the post-tests and they were not 
allowed to use their dictionaries during test administration. Besides, before the administration 
of the study the students were reminded that participation was voluntary and there would be 
no extra marks or rewards. The collection of all surveys of Group A and B took one class 
hour - 45 minutes. The watching of the episodes also lasted nearly 40 minutes for each group. 
Afterwards, the participants were given the VKS and were asked to answer the items on it. 
The expected responses were placed on a five-point scale indicated above. 

Finally, the participants’ responses were loaded into SPSS 15.0. Then, descriptive 
statistics on SPSS were employed by the researcher to see the general distribution of the data 
and average scores (Table 2). Following descriptive statistics, one sample t-test was used to 
see the progress in each group (Table 3). Further analysis was conducted to understand 
whether one of the groups improved significantly better than the other. For this purpose, 
independent sample t-test was utilized (Table 4). 

5. Discussion 

According to the results of the descriptive statistics (see Table 2), the highest mean value was 
seen in the post-test of Group A (Mean= 2.81) and the second highest mean value lies in the 
post-test of Group B. Also, the mean scores of pre-tests of both groups were very close to 
each other (Group A= 2.47, Group B= 2.53). It is assumed, thus, that both groups had similar 
knowledge about the target words before they were exposed to the treatment. 
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Table 2: Average scores attained in each group. 



N 

Minimum 

Maximum 

Mean 

SD 

Group Pretest 

21 

1.05 

4.27 

2.4735 

1.0400 

Group Posttest 

21 

1.11 

4.88 

2.8148 

1.1834 

Group Pretest 

21 

1.05 

4.11 

2.4232 

.9815 

Group Posttest 

21 

1.11 

4.50 

2.5343 

1.0989 

Valid N 

21 






(listwise) 


The development in each group was measured through t-test. One sample T-test demonstrated 
that there was an improvement in each group. Participants in Group A improved on an 
average of .34127 from pre-test to post-test while those in Group B (the subtitles group) 
progressed by .1111 approximately. As Table 3 illustrates, the progress made in each group 
was significant at a .01 level. 


Table 3: Summary of one-sample r-test results. 



T 

Df 

Sig. (2- 

Mean 

95% Confidence Interval 




tailed) 

Difference 

of the Difference 






Upper 

Lower 

Gains for 

Group A 

11.391 

20 

.000 

.34127 

.4066 

.2751 

Gains for 

Group B 

10.985 

20 

.000 

.11111 

.1645 

.0577 


*significant at p < .01 


Further analysis was conducted through t-test to discover whether one of the groups improved 
significantly more than the other (Table 4). Prior to this, the homogeneity variances were 
checked through Levene statistic and it showed no significance (p=521). This indicated that 
there was no evidence of homogeneity. Though Group A slightly outperformed Group B on 
average scores, the t-test results demonstrated that there was no statistically significant 
difference between the two groups’ gains (p=.094). 
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Table 4: The summary of t-test results for the gains between two groups (Independent Samples Test). 



F 

Sig. 

t 

df 

Sig. (2- 

taied) 

Mean 

difference 

Std. Error 
difference 

95 % 

Confidence 
Interval of the 
Differences 








Lower 

Upper 

Mean Equal 










variances 

.419 

.521 

1.715 

40 

.094 

.28042 

.16350 

-.05002 

.61086 

assumed 










Equal 










variances 

not 



1.715 

39.292 

.094 

.28042 

.16350 

-.05020 

.61105 

assumed 











According to the findings of the study, both Group A (the subtitle group) and Group B 
(the no-subtitle group) had significant gains from pre-test to post-tests in the self-reported 
Vocabulary Knowledge Scales. When the gains of two groups were compared, the 
participants in Group A, who watched cartoons with subtitles, were found to improve a bit 
more than those in Group B, who watched without subtitles (Group A= .34127, Group B= 
.11111). However, there was no significant difference between groups in terms of gains. 
Thus, what facilitated the improvement in vocabulary knowledge was not the incorporation of 
subtitles into the cartoons. At this point, it might be posited that the cartoons increased the 
vocabulary development of participants. 

As mentioned earlier, both groups’ initial knowledge was close to each other before 
the treatment (see pre-test scores of the groups). The mean scores were slightly under 2.5. on 
average. This was an indication that most participants had seen the words before but did not 
remember what they meant. However, the post-test points of both groups increased 
concurrently after watching the cartoons (Group A from 2.47 to 2.81, Group B from 2.42 to 
2.53). Based on this fact, it might be assumed that the participants not only remembered 
seeing the words but also guessed almost half of the target words accurately by providing 
their Turkish equivalents or translations. 

The previous research attributes the vocabulary development of students to their being 
exposed to the target words in a specific context (in subtitled cartoons). Nation and Waring 
(1997) point out that it is one of the most important vocabulary learning strategies and an 
essential part of any vocabulary learning program. Since the participants were not informed 
about the purpose of the study beforehand and were not allowed to use their dictionaries 
during the treatment phase, they most probably took advantage of this strategy with the help 
of contextual clues embedded in the cartoons. Based on this fact, it is assumed that incidental 
learning of the vocabulary items occurred due to the incorporation of target words into the 
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cartoons that functioned as a context, obviously a fundamental notion within the process of 
incidental vocabulary learning (DeRidder, 1999). It is also preferred by teachers since 
teaching words in isolation does not produce the desired results. 

The data collection tool utilized in this study, VKS, is concerned with the descriptions 
of stages that words pass through. That is why the use of such scales seems attractive on the 
surface. However, such scales have been criticized in many ways. For example, Meara (1996) 
criticizes Wesche and Parikbath’s VKS by claiming that the description of the word 
knowledge is rather simple; only the very basic stages that a vocabulary item passes through 
are defined and there is no attempt to account for more detailed knowledge about a word that 
develops over time. However, it was not an obstacle for the study to use VKS since its goal 
was not to account for detailed knowledge about the target words or more complex stages 
they pass through. Instead, the main aim was to identify the initial stages or levels in 
vocabulary in vocabulary development by students’ self-reports and demonstrations. To the 
researcher’s understanding, VKS satisfied the objectives of the study. 

The materials used in the treatment phase, namely cartoon movies, attracted students’ 
attention. They seemed to enjoy watching the episodes. However, it was observed by the 
researcher that a few of the students lost their interest while watching the second cartoon 
movie. The reason was attributed to the length of the cartoon, nearly 40 minutes in total. 
Given the concentration span of students, it would have been better if the movies had been 
kept under 30 minutes. Thus, in further research, the cartoons should be trimmed to include 
only the related parts targeted in the study. 

6. Conclusion 

In this study, the relationship between watching subtitled and non-subtitled cartoons and its 
effect on the vocabulary development was investigated. It has been found that, no matter 
whether participants watched the movies with subtitles or without them, they improved at a 
significant level from pre-test to post-test scores. The gains of participants were thought to be 
the result of contextual aids of cartoons. The actions, signals of hands and arms, as well as 
facial expressions might facilitate the understanding of the target verbs when accompanied 
with subtitles. Finally, the following propositions can be concluded from this study: 

1. Vocabulary development is a long lasting process that needs to be supported by contextual 
clues. It is due to the fact that the possibility of guessing meaning from context is higher. 
Words in isolation, overall, give no clues to the learner and thus they are hard to guess from 
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context. Then, the implication for the teachers is that the new vocabulary items should be 
presented in context rather than in isolation. Nevertheless, it is worth noting that this 
conclusion would need to be verified by a longitudinal study. Thus, one of the limitations of 
this study was that the interval between two tests was too short, which might have caused bias 
in the interpretation of the results. 

2. Frequent encounters with novel words make learners become familiarized with the words, 
which was supported by the findings of the study. Those students who claimed in the pre-test 
that they had not seen the word before reported in the post-test that they could make a guess 
about the unknown word. This familiarization was definitely strengthened by repetition. 
Consequently, vocabulary improvement occurred incidentally regardless of watching the 
episodes with or no subtitles. 

3. Though there were no statistically significant differences between subtitle and no subtitle 
groups in terms of vocabulary gains, the mean score was higher in the subtitle group from 
pre-test to post-test. The lack of homogeneity in groups might be the reason of this high mean 
score. To prevent such kind of random errors, the group members should have taken the 
homogeneity test before being assigned to any group. 

4. One implication for further research is that other word groups (i.e. adjectives or nouns) can 
be selected to work on with a focus on different language skills (i.e. listening, reading 
comprehension, etc.) Another idea might be to incorporate only the keywords as subtitles 
since the whole subtitled sentences make it difficult for learners to follow the flow of speech, 
as they either concentrate on the speech or the subtitles. Whenever this happens, it becomes 
hard to measure the vocabulary gains. Therefore, including only the target words in the form 
of keywords may bring us closer to the goal. 
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Appendix 


Dear student. 

This questionnaire was prepared to measure your knowledge of listed words. The answers to the survey will be 
used only in accordance with research objectives and will be kept confidential . Sincere answers to the questions 
are of high significance for the success and reliability of the study. Thank you very much for taking your time to 
help me. 


Vocabulary Knowledge Scale 


1: I don't remember having seen this word before 

2: I have seen this word before but I don't know what it means 

3: I have seen this word before and I think it means ('svnonvm or translation) 

4: I know this word. It means ('svnonvm or translation) 

5: I can use this word in a sentence, e.g.: (if vou do this section, please also do section 

Mark the appropriate column for each word and provide an answer, if necessary. 

Note: You can write synonyms, translations and sentences in Turkish, if you wish. 

VERBS 

LEVELS 

1 

2 

3 

4 

5 

1. mop 






2. hug 






3. hail 






4. sigh 






5. fart 






6. gasp 






7. swab 






8. dump 






9. rely 






10. ditch 






11. growl 






12. vomit 






13. roast 






14. murmur 






15. chuckle 






16. applaud 






17. negotiate 






18. terminate 







































